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Objectives 

A  key  part  of  building  trustworthy  software  systems  is  having  some  basis  to 
believe  that  the  system  will  behave  as  expected  in  the  presence  of  attacks 
and/or  failures,  as  well  as  design  and  programming  errors.  And  any  basis  for 
having  trust  in  a  computing  system  will  ultimately  be  grounded  in  human 
comprehension,  for  only  if  we  can  understand  and  predict  how  an  artifact 
behaves  can  we  trust  it.  Humans  are  limited  in  what  they  can  understand, 
though.  So  research  to  facilitate  creating  trustworthy  systems  that  are  large 
and  complex  must  somehow  provide  a  means  to  amplify  trust  in  artifacts 
that  humans  can  understand.  We  need  the  means  to  leverage  components 
we  do  trust  and  the  means  to  transfer  trust  from  one  part  of  a  system  to 
another. 

The  objective  of  this  project  was  to  investigate  two  particularly  promis¬ 
ing  avenues  for  trust  amplification,.  Both  avenues  built  on  unexplained  but 
nevertheless  empirically  observed  asymmetries — one  between  proof  genera¬ 
tion  and  proof  checking,  the  other  between  public  key  encryption  and  public 
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key  decryption: 

•  Taking  language-based  security  the  next  step,  by  further  developing 
the  Cyclone  type-safe  variant  of  C  and  by  marrying  certifying  compila¬ 
tion  with  program  analyzers  and  IRM  rewriters  as  a  means  of  making 
language-based  enforcement  technologies  more  trustworthy. 

•  Combining  proactive  secret-sharing  and  threshold  cryptography  in  con¬ 
nection  with  quorums  for  replication-management  as  a  means  of  imple¬ 
menting  trustworthy  services  in  settings  satisfying  only  weak  assump¬ 
tions. 


Accomplishments 

Cyclone,  Implementation  errors,  such  as  buffer  overruns,  memory  leaks, 
and  integer  overflows  account  for  an  alarming  number  of  successful  attacks. 
To  address  these  concerns,  the  project  developed  a  type-safe  variant  of  C 
called  Cyclone.  The  Cyclone  language  retains  the  familiar  syntax  and  se¬ 
mantics  of  C  code,  so  that  legacy  programs  can  be  easily  ported  to  an  envi¬ 
ronment  with  the  same  strong  security  guarantees  of  modern  languages  such 
as  Java. 

The  original  version  of  Cyclone  used  a  combination  of  static  type-checking 
and  run-time  tests  to  detect  and  prevent  implementation  errors.  To  minimize 
run-time  overhead — and  to  decrease  the  probability  of  run-time  failure — 
this  project  extended  the  compiler  with  support  for  extended  static  checking, 
(ESC). 

The  ESC  component  of  the  Cyclone  compiler  was  first  used  to  eliminate 
run-time  checks  that  could  not  be  eliminated  through  the  static  type  system, 
including  checks  on  array  subscripts,  pointer  arithmetic,  etc.  This  was  ac¬ 
complished  by  first  constructing  a  verification  condition^-a,  logical  assertion 
whose  validity  ensures  that  the  run-time  check  will  never  fail.  The  verification 
conditions  for  each  check  were  then  fed  to  a  custom  theorem  prover.  When 
the  theorem  prover  could  successfully  prove  the  verification  condition,  the 
corresponding  check  was  eliminated.  When  the  prover  could  not  discharge 
the  verification  condition,  a  warning  was  issued  to  the  programmer. 

The  primary  challenges  in  this  effort  were  (i)  making  the  verification  con¬ 
dition  generation  scalable  and  precise  and  (ii)  finding  fast  but  sound  heuris¬ 
tics  for  the  theorem  prover.  For  instance,  previously  proposed  algorithms 
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could  generate  verification  conditions  exponentially  larger  than  the  original 
source  code.  The  project  developed  a  new  algorithm  that  ensures  the  condi¬ 
tions  are  quadratic  in  the  size  of  the  source  in  the  worst  case  and  linear  in 
practice. 

Once  these  hurdles  were  overcome,  the  compiler  was  able  to  automatically 
eliminate  96%  of  the  run-time  checks  in  the  roughly  80,000  lines  of  Cyclone 
code  that  make  up  the  standard  libraries  and  compiler.  Furthermore,  the 
additional  analyses  and  transformations  only  added  2%  overhead  to  compile 
times. 

Inlined  Reference  Monitors.  In  earlier  work,  PI  Schneider  developed  a 
model  of  enforceable  security  policies  and  showed  that,  in  principle,  all  such 
policies  could  be  realized  using  some  form  of  a  reference  monitor.  A  reference 
monitor  observes  system  execution  and  blocks  actions  that  would  violate  a 
desired  security  policy.  Today,  most  reference  monitors  are  implemented 
by  an  operating  system  using  hardware-enforced  primitives.  However,  the 
class  of  policies  that  can  be  directly  enforced  using  hardware  mechanisms 
is  severely  limited  because  the  machine  has  no  knowledge  of  application- 
level  semantics,  only  limited  opportunity  to  intervene,  and  few  provisions  for 
extending  policies  to  cover  new  situations. 

To  overcome  these  limitations,  a  more  general  class  of  enforcement  mech¬ 
anisms  based  on  inlined  reference  monitors  (IRMs)  was  studied  under  the 
auspices  of  this  funding.  An  IRM  tool  takes  a  policy  and  rewrites  untrusted 
code,  inserting  checks  and  actions  that  ensure  the  resulting  code  will  re¬ 
spect  the  policy  when  executed.  Inlining  the  monitor’s  code  allows  it  to  have 
complete  access  to  the  internals  of  the  application,  and  allows  it  to  tailor 
the  policy  to  the  application  and  its  abstractions.  Furthermore,  the  IRM  ap¬ 
proach  makes  it  possible  to  insert  corrective  actions  instead  of  simply  halting 
the  application. 

Building  on  Schneider’s  earlier  work,  the  project  developed  a  more  re¬ 
fined  characterization  of  what  policies  can  be  enforced  using  inlined  reference 
monitors.  Specifically,  the  Pis  developed  a  model  based  on  standard  Turing 
machines,  adapted  Schneider’s  criteria  for  enforceable  security  policies,  and 
introduced  computability  requirements.  Static  analysis  and  classical  refer¬ 
ence  monitors  were  also  integrated  into  the  model.  This  allowed  comparing 
the  relative  power  of  the  various  enforcement  mechanisms  and  relating  the 
power  of  these  mechanisms  to  standard  computability  results.  For  instance, 
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it  was  relatively  easy  to  show  that  the  class  of  policies  precisely  supported 
by  static  analysis  could  also  be  supported  by  both  classical  and  inlined  ref¬ 
erence  monitors.  In  addition,  introducing  a  computability  requirement  on 
reference  monitors  was  found  to  be  necessary,  but  not  sufficient,  for  precise 
characterization  of  the  class  of  policies  they  can  actually  realize.  And  a  new 
property,  called  “benevolence”  was  identified;  it  provides  a  more  accurate 
upper  bound  on  the  power  of  reference  monitors. 

The  most  surprising  and  important  results  obtained  concern  the  general 
framework  of  IRMs.  The  class  of  policies  originally  characterized  by  Schnei¬ 
der  was  shown  not  to  include  all  policies  enforceable  through  IRMs  (and  vice 
versa).  Indeed,  the  class  of  policies  enforceable  through  IRMs  was  shown 
not  to  correspond  to  any  class  of  the  Kleene  hierarchy.  This  is  a  surprising 
and  an  important  result,  because  it  shows  that  inlined  reference  monitors  are 
truly  a  powerful  security  enforcement  technique. 

In  addition  to  a  theoretical  study  of  IRMs,  a  number  of  practical  issues 
were  studied  too.  For  instance,  to  achieve  acceptable  performance,  an  IRM 
rewriter  needs  to  perform  a  number  of  sophisticated  optimizations  so  that 
checks  and  IRM  state  updates  are  inserted  only  where  needed.  Furthermore, 
an  IRM  rewriter  must  be  careful  to  ensure  that  the  integrity  of  the  reference 
monitor  cannot  be  violated  by  untrusted  code.  In  practice,  this  means  IRM 
rewriters  are  relatively  large  tools  (similar  to  compilers)  that  undoubtably 
contain  bugs,  and  thus  should  not  be  trusted. 

To  address  this  concern,  the  project  combined  the  ideas  behind  proof¬ 
carrying  code  (PCC)  with  IRM  rewriting  and  developed  a  certified  IRM 
rewriter  framework  for  Microsoft  .NET  code.  In  this  framework,  the  rewriter 
produces  explicit  evidence  that  enables  an  independent  proof  checker  to  de¬ 
termine  that  the  rewritten  code  respects  the  desired  security  policy.  The 
proof  checker  is  relatively  small  compared  to  rewriters  and  is  therefore  likely 
to  be  more  trustworthy. 

The  evidence  that  the  rewriter  provides  to  the  checker  is  in  the  form  of 
extended  typing  annotations.  A  model  of  the  .NET  intermediate  language 
was  constructed  along  with  a  proof  that,  given  proper  evidence  that  the  pro¬ 
gram  type-checks  (under  the  extended  type  system),  any  execution  sequence 
of  the  code  will  respect  the  policy. 

Implementing  Trustworthy  Services.  To  be  trustworthy ,  a  service  must 
tolerate  failures  and  attacks.  The  means  to  build  fault-tolerant  services  has 
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been  at  hand  for  some  time — replication  of  servers  on  independent  hosts. 
Scaling  that  up  to  handle  attacks  was  a  third  research  focus  of  this  grant. 
In  particular,  although  server  failures  are  often  observed  to  be  independent, 
independence  in  the  presence  of  attacks  is  typically  not  seen.  An  adversary 
that  disrupts  one  replica  will  probably  be  able  to  exploit  the  same  vulner¬ 
ability  at  other  replicas  and  disrupt  them  as  well — having  2/  +  1  replicas 
might  tolerate  up  to  /  faulty  hosts,  whereas  all  hosts  (hence  all  replicas) 
would  succumb  to  a  single  attack. 

Server  diversity  thus  is  central  to  implementing  trustworthy  services  with 
replicated  servers.  Such  diversity  can  be  introduceed  automatically  during 
compilation,  loading,  or  in  the  run-time  environment  by  using  an  obfuscator, 
which  transforms  a  base  program  into  a  morph  according  to  some  semantics¬ 
preserving  transformations.  Different  subsets  of  these  transformations  yield 
different  morphs,  with  many  of  the  transformations  themselves  being  non- 
deterministic.  An  obfuscation  key  input  to  the  obfuscator  determines  the 
exact  transformations  used  for  computing  a  specific  morph,  and  two  different 
obfuscation  keys  are  likely  to  produce  unpredicatably  different  morphs  of  the 
same  base  program. 

Use  of  an  obfuscation  key  not  known  by  attackers  defends  against  attacks 
that  work  by  exploiting  detailed  knowledge  of  base  server  code.  But  over 
time,  details  of  a  morph’s  code  could  become  known  to  attackers,  so  simply 
running  different  morphs  on  each  server  is  not  sufficient.  This  is  just  the  well 
known  failing  of  the  oft-maligned  “security  by  obscurity” .  There  is,  however, 
a  promising  defense  available — proactive  execution  of  obfuscation,  whereby 
each  server  periodically  selects  a  fresh  (secret)  obfuscation  key,  computes  a 
new  morph,  and  executes  that  morph  for  its  next  window  of  vulnerability. 

Two  classes  of  questions  had  to  be  addressed  in  getting  proactive  obfusca¬ 
tion  to  work;  that  drove  the  research.  First,  were  questions  about  protocols 
to  ensure  that  morphs  would  be  terminated  and  restarted  in  a  way  that  re¬ 
sisted  compromise  by  attackers.  The  needs  here  were  not  being  addressed 
by  extant  agreement  protocols,  because  morphs  have  their  storage  purged 
and  reloaded  (to  eliminate  undetectably  compromised  code  and  data)  at  the 
start  of  each  window  of  vulnerability.  Not  only  were  such  amnesia-insensitive 
agreement  protocols  developed  as  part  of  this  project,  but  a  firewall  that  em¬ 
ploys  proactive  obfuscation  was  designed  and  prototyped.  Having  the  pro¬ 
totype  allowed  different  protocols  to  be  explored  and  ensured  that  protocol 
design  was  grounded  in  the  realities  of  an  actual  system  context. 

The  second  class  of  problems  concerned  foundations  of  the  approach: 
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What  classes  of  attacks  would  be  blunted  by  proactive  obfuscation?  Specific 
correspondences  between  classes  of  transformations  and  classes  of  attacks 
they  defend  against  was  known  at  the  start  of  this  project.  But  although 
knowing  this  correspondence  is  important  when  designing  a  set  of  defenses 
for  a  given  threat  model,  knowing  the  specific  correspondences  is  not  the 
same  as  knowing  the  overall  power  of  mechanically-generated  diversity  as  a 
defense.  This  project  explored  that  latter,  broader,  issue,  by 

•  giving  a  semantics  for  proving  results  about  the  defensive  power  of 
obfuscation; 

•  giving  a  precise  characterization  of  attacks,  applicable  to  viewing  di¬ 
versity  as  a  defense; 

•  developing  the  thesis  that  mechanically-generated  diversity  is  compa¬ 
rable  to  type  systems,  and  deriving  an  admittedly  unusual  type  system 
equivalent  to  obfuscation  in  the  presence  of  finitely  many  keys; 

•  exhibiting,  for  a  C-like  language  and  non-trivial  obfuscator,  an  increas¬ 
ingly  tighter  sequence  of  type  systems  for  soundly  approximating  ob¬ 
fuscation  under  arbitrary  finite  sets  of  keys.  Surprisingly,  the  more 
accurate  type  systems  are  based  on  information  flow.  No  type  system 
corresponds  exactly  to  the  obfuscator  under  arbitrary  finite  sets  of  keys, 
and  therefore  approximations  are  the  best  that  can  be  achieved. 
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20.  Greg  Morrisett,  Matthew  Fluet,  and  Amal  Ahmed.  L 3:  A  linear 
language  with  locations.  Seventh  International  Conference  on  Typed 
Lambda  Calculi  and  Applications  (TLCA’05),  (Nara,  Japan,  April  2005), 
293-307. 
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21.  Scott  D.  Stoller  and  Fred  B.  Schneider.  Automated  analysis  of  fault- 
tolerance  in  distributed  systems.  Formal  Methods  in  System  Design 
28,  2  (March  2005),  183-196. 

22.  Fred  B.  Schneider.  It  depends  on  what  you  pay.  Editorial.  IEEE 
Security  and  Privacy  3,  3  (May/ June  2005),  5. 

23.  Michael  R.  Clarkson,  Andrew  C.  Myers,  and  Fred  B.  Schneider.  Be¬ 
lief  in  Information  Flow.  Proceedings  18th  IEEE  Computer  Security 
Foundations  Workshop  (Aix-en-Provence,  France,  June  20-22,  2005), 
31-45. 

24.  Lidong  Zhou  and  Fred  B.  Schneider.  APSS:  Proactive  secret  sharing  in 
asynchronous  systems.  ACM  Transactions  on  Information  and  System 
Security  8,  3  (August  2005),  1-28. 

25.  Amal  Ahmed,  Matthew  Fluet,  and  Greg  Morrisett.  A  step-indexed 
model  of  substructural  state.  In  Proceedings  of  the  ACM  International 
Conference  on  Functional  Programming  (ICFP’05),  (Tallinn,  Estonia, 
September  2005),  78-91. 

26.  Lidong  Zhou  and  Fred  B.  Schneider.  Implementing  trustworthy  services 
using  replicated  state  machines.  IEEE  Security  and  Privacy,  Volume 
3,  Number  5  (September/October  2005),  34-43. 

27.  Matthew  Fluet  and  Greg  Morrisett.  Monadic  regions.  Journal  of  Func¬ 
tional  Programming,  to  appear. 


Interactions/Transitions 

•  Schneider  is  the  director  of  the  AFRL/ Cornell  Information  Assurance 
Institute. 

•  For  2003,  Schneider  served  as  Chief  Scientist  of  the  Griffiss  Institute, 
New  York  State  Cybersecurity  consortium,  and  served  on  the  Board  of 
Directors  for  the  following  2  years. 

•  Schneider  served  as  a  member  of  the  following  industrial  advisory  boards: 
CIGITAL  Technical  Advisory  Board;  Fast  Search  and  Transfer  Techni¬ 
cal  Advisory  Board;  IBM  Corporation  Autonomic  Computing  Advisory 
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Board;  Intel  Microprocessor  Research  Lab  Advisory  Board;  Microsoft’s 
Trustworthy  Computing  Academic  Advisory  Board;  Packet  General 
Networks  Technical  Advisory  Board. 

•  Schneider  served  on  the  following  other  advisory  committees:  UK  De¬ 
pendability  Interdisciplinary  Research  Collaboration  (DIRC),  Steer¬ 
ing  Committee;  ACM  Advisory  Committee  on  Security  and  Privacy 
(ACSP);  National  Research  Council  Computer  Science  and  Telecom¬ 
munications  Board;  CSTB  study  Committee  on  Improving  Cybersecu¬ 
rity  in  the  U.S;  NSF/CISE  Advisory  Committee. 

•  Schneider  served  on  the  editorial  boards  for:  Distributed  Computing, 
Information  Processing  Letters ,  High  Integrity  Systems,  Annals  of  Soft¬ 
ware  Engineering,  ACM  Computing  Surveys.  He  is  also  co-managing 
Editor  for  Springer- Verlag’s  Texts  and  Monographs  in  Computer  Sci¬ 
ence  and  Associate  Editor-in-Chief  for  IEEE  Security  and  Privacy  mag¬ 
azine. 

•  Schneider  is  Chief  Scientist  of  the  NSF  TRUST  Science  and  Technol¬ 
ogy  Center,  which  includes  U.C.  Berkeley,  Carnegie-Mellon  University, 
Cornell  University,  Stanford  University,  and  Vanderbilt  University. 

•  Morrisett  serves  on  Microsoft’s  Trustworthy  Computing  Academic  Ad¬ 
visory  Board  and  the  Fortify  Technical  Advisory  Board. 

•  Morrisett  serves  on  the  editorial  boards  for:  Journal  of  Functional  Pro¬ 
gramming  and  ACM  Transactions  on  Programming  Languages  and  Sys¬ 
tems.  He  also  serves  on  the  Advisory  Committee  for  the  Semantics,  Ap¬ 
plications,  and  Implementations  of  Program  Generation  (SAIG)  con¬ 
ference;  and  the  ACM  Conference  on  Types  in  Language  Design  and 
Implementation. 

•  Morrisett  served  as  the  program  chair  for  the  2003  ACM  Symposium 
on  Principles  of  Programming. 

DoD  Interactions  and  Technology  Transitions 

•  As  a  consultant  to  DARPA/IPTO,  Schneider  chaired  the  indepen¬ 
dent  evaluation  team  for  the  OASIS  Dem/Val  prototype  project.  This 
project  funded  the  design  of  a  JBI  system  intended  to  tolerate  a  class 
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A  Red  Team  attack  for  12  hours.  Schneider  also  served  on  the  indepen¬ 
dent  evaluation  team  for  the  DARPA/IPTO  Self- Regenerative  Systems 
program. 

•  Schneider  served  twice  on  AFRL  search  committees  for  Senior  Scientists 
in  Information  Assurance  Technology. 

•  Morrisett  and  Schneider  each  briefed  the  Infosec  Research  Council’s 
“Research  Hard  Problems”  study;  Schneider  also  served  as  a  reviewer 
for  the  final  resport. 

•  Morrisett  worked  on  the  development  of  Microsoft’s  tools  for  automat¬ 
ically  finding  security  flaws  in  production  code,  based  on  his  experi¬ 
ence  with  Cyclone.  He  and  Schneider  also  worked  with  student  Kevin 
Hamlen  and  Microsoft  researchers  on  the  implementation  of  the  .NET 
rewriting  tool  for  inline  reference  monitors. 

•  AT&T  research  worked  with  Morrisett  to  develop  the  Cyclone  language, 
compiler,  and  tools.  In  addition,  researchers  at  the  University  of  Mary¬ 
land,  the  University  of  Utah,  Princeton,  the  University  of  Washington, 
the  University  of  Pennsylvania,  and  Cornell  are  all  using  Cyclone  to 
develop  research  prototypes. 

Honors  and  Awards:  7/2003  -  11/2005 

F.B.  Schneider: 

•  Doctor  of  Science  ( honoris  causa),  University  of  NewCastle-upon-Tyne 
(2003). 
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